NLPGD
Preface
Employees are the heart and soul of a business. Their actions and performance are what drive the profitability of a business and it’s ability to maximize shareholder value. In the following document, we look to address the following question:
Is there a relationship between employee sentiment and excess returns?
By building a webscraping program and applying it to GlassDoor, we were able to compile thousands of employee reviews across a variety of companies:
- ExxonMobil (XOM)
- Chevron (CVX)
- Shell (SHEL)
- BP (BP)
For context on the analysis that occurs below, here is a chart that represents their respective share prices from the last 10 years:
We wanted to select companies that all operate in the same industry and differ by market cap. This ensures that our analysis truly represents differentiating factors in relation to excess returns.
Firstly, we needed to make sense of all the reviews and how best to compile and analyze them. Natural Language processing is the computational analysis of language and speech. This essentially analyzes words and phrases and rates them numerically or categorically based off of the emotion and message they are conveying. Here is a short description of the two methods we applied:
- Bing: A dataset of 6,786 words with binary positive and negative sentiment scores
library(gt)
p <- bing %>% slice_head(n = 3)
n <- bing %>% slice_tail(n = 3)
b <- rbind(p, n)
b %>% gt() %>% tab_options(
data_row.padding = px(6),
heading.align = 'left',
column_labels.background.color = 'dodgerblue4',
heading.title.font.size = 26,
footnotes.font.size = 8
) %>%
tab_style(style = cell_text(color = 'dodgerblue4',
weight = 'bold'),
locations = cells_title(groups = 'title')) %>% tab_options(table.font.size = 10, heading.title.font.size = 15)| word | sentiment |
|---|---|
| 2-faces | negative |
| abnormal | negative |
| abolish | negative |
| zest | positive |
| zippy | positive |
| zombie | negative |
- Afinn: A dataset of 2477 words with scores ranging from -5 to 5:
ap <- afinn %>% slice_head(n = 500)
an <- afinn %>% slice_tail(n = 500)
a <- rbind(ap,an) %>% arrange(desc(value)) %>% slice_head(n = 6)
a %>% gt() %>% tab_options(
data_row.padding = px(6),
heading.align = 'left',
column_labels.background.color = 'dodgerblue4',
heading.title.font.size = 26,
footnotes.font.size = 8
) %>%
tab_style(style = cell_text(color = 'dodgerblue4',
weight = 'bold'),
locations = cells_title(groups = 'title')) %>% tab_options(table.font.size = 10, heading.title.font.size = 15)| word | value |
|---|---|
| breathtaking | 5 |
| superb | 5 |
| thrilled | 5 |
| amazing | 4 |
| awesome | 4 |
| brilliant | 4 |
We also utilized the star ratings that employees leave whenever a review is posted (ranging from 1 star to 5 stars). To organize our data into workable metrics. We used individual words instead of ngrams (collection of words or short sentences) as the bulk of the reviews are short and sometimes contain only one word. To summarize, we opted for the following:
- Grouped reviews into monthly, quarterly and annual bins.
- Bing was organized into percentage of reviews that were deemed positive and averaged over monthly, quarterly and annually.
- Afinn was organized into average total score over monthly, quarterly and annually.
- Star ratings were averaged over monthly, quarterly and annually.
Here are some visuals that are separated by company, depicting how these various metrics change through time. All charts use monthly data to help tell the story:
We also utilized the star ratings that employees leave whenever a review is posted (ranging from 1 star to 5 stars). To organize our data into workable metrics. We used individual words instead of ngrams (collection of words or short sentences) as the bulk of the reviews are short and sometimes contain only one word. To summarize, we opted for the following:
- Grouped reviews into monthly, quarterly and annual bins.
- Bing was organized into percentage of reviews that were deemed positive and averaged over monthly, quarterly and annually.
- Afinn was organized into average total score over monthly, quarterly and annually.
- Star ratings were averaged over monthly, quarterly and annually.
Here are some visuals that are separated by company, depicting how these various metrics change through time. All charts use monthly data to help tell the story:
review_comb %>% plot_ly(x = ~month,
y = ~meanRateMon,
color = ~company,
type = 'scatter',
mode = 'lines') %>% layout(title = "Average Monthly Star Reviews",
xaxis = list(title = ""),
yaxis = list(title = "Stars"))Alpha Analysis
Using CAPM, Alphas were calculated annually for each individual stock
- 1 Year Beta
- S&P500 utilized as a proxy for market return
Annual Alphas were regressed annually against the preceding year’s average review ratings and sentiment, as defined by the Affin dictionary.
- Bing NLP excluded due to model multicollinearity
Model fit was poor for all four companies
Alpha between the four firms w
prop_postive mean_rating sentiment
2.770702 3.536015 5.855219
Call:
lm(formula = alpha ~ prop_postive + mean_rating + sentiment,
data = comb_y)
Residuals:
Min 1Q Median 3Q Max
-0.32730 -0.11535 0.06123 0.11206 0.19272
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -0.4098 0.5879 -0.697 0.491
prop_postive 0.6435 0.5910 1.089 0.284
mean_rating 0.1438 0.2297 0.626 0.536
sentiment -0.2323 0.2631 -0.883 0.384
Residual standard error: 0.1573 on 32 degrees of freedom
Multiple R-squared: 0.04484, Adjusted R-squared: -0.04471
F-statistic: 0.5008 on 3 and 32 DF, p-value: 0.6844
Call:
lm(formula = alpha ~ mean_rating + sentiment, data = comb_bp)
Residuals:
Min 1Q Median 3Q Max
-0.33669 -0.01916 0.02254 0.09638 0.18503
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -0.1752 1.3793 -0.127 0.903
mean_rating -0.0846 0.4620 -0.183 0.861
sentiment 0.2999 0.3961 0.757 0.478
Residual standard error: 0.1749 on 6 degrees of freedom
Multiple R-squared: 0.1046, Adjusted R-squared: -0.1939
F-statistic: 0.3504 on 2 and 6 DF, p-value: 0.7179
mean_rating sentiment
1.604489 1.604489
mean_rating sentiment
3.080539 3.080539
Call:
lm(formula = alpha ~ mean_rating + sentiment, data = comb_exxon)
Residuals:
Min 1Q Median 3Q Max
-0.22321 -0.08722 -0.04044 0.12274 0.19273
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -1.6868 1.3571 -1.243 0.26
mean_rating 0.8148 0.5521 1.476 0.19
sentiment -0.6364 0.4879 -1.304 0.24
Residual standard error: 0.1595 on 6 degrees of freedom
Multiple R-squared: 0.2686, Adjusted R-squared: 0.02485
F-statistic: 1.102 on 2 and 6 DF, p-value: 0.3912
Call:
lm(formula = alpha ~ mean_rating + sentiment, data = comb_shell)
Residuals:
Min 1Q Median 3Q Max
-0.32304 -0.08390 0.05365 0.11198 0.15678
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 0.057339 1.731574 0.033 0.975
mean_rating 0.010772 0.694483 0.016 0.988
sentiment -0.004384 0.734549 -0.006 0.995
Residual standard error: 0.1863 on 6 degrees of freedom
Multiple R-squared: 5.874e-05, Adjusted R-squared: -0.3333
F-statistic: 0.0001762 on 2 and 6 DF, p-value: 0.9998
mean_rating sentiment
2.739596 2.739596
Portfolio-based Analysis
Two portfolios of our selected companies were created
- Portflio 1: Naive equally-weighted portfolio
- Rebalanced yearly based upon the prior years closing price
- Portfolio 2: Optimized portfolio utilizing three sentiment indicators
- Generated weights utilizing a combination the input indicators
- Optimization process to determine the ideal relative sentiment mix
- Training window: 2014-2021
- Rebalanced annually using an aggregation sentiment data from the prior year
- Sentiment is treated on a absolute basis as opposed to on a relative basis to prior years
- Companies are proportional de-weighted in cases of ‘review slippage’ relative to their peers
- No additionally compounds are included for +/- review drift
# A tibble: 1 × 5
logRet_sum var prop rate sent
<dbl> <dbl> <dbl> <dbl> <dbl>
1 0.424 0.0683 0 1 0
# A tibble: 1 × 2
logRet_sum var
<dbl> <dbl>
1 0.428 0.0739
#```{r}
Ignore below for the time being
port_y <- adj_price %>% filter(date >= ‘2013-12-31’, date <= ‘2024-01-01’) %>% mutate(company = case_when(Symbol == “BP” ~ “bp”, Symbol == “CVX” ~ “Chevron”, Symbol == “SHEL” ~ “Shell”, Symbol == “XOM” ~ “ExxonMobil”)) %>% mutate(year = year(date), y = as.numeric(paste0(year, quarter(date))), m = paste0(year, month(date))) %>% group_by(y) %>% filter(date == max(date)) %>% ungroup() %>% #subtract a year to bind correct lagged sentiment mutate(y = y - 1) %>% group_by(company) %>% mutate(lag_close = lag(Close)) %>% drop_na()
rev_y <- rev_base %>% group_by(q, company) %>% select(-y) %>% rename(y = q) %>% mutate(y = as.numeric(y)) %>% summarise(mean_rating = mean(ratingValue), count = n())
affin_y <- affin_base %>% group_by(company, q) %>% select(-y) %>% rename(y = q) %>% mutate(y = as.numeric(y)) %>% summarise(sentiment = mean(value))
bing_y <- bing_base %>% group_by(company, q) %>% select(-y) %>% rename(y = q) %>% mutate(y = as.numeric(y)) %>% summarise(prop_postive = mean(p_n))
indicators_y <- full_join(rev_y, affin_y, by = c(“company”, “y”)) indicators_y <- full_join(bing_y, indicators_y, by = c(“company”, “y”))
#joining prices at year end with sentiment port_y <- full_join(port_y, indicators_y, by = c(“company”, “y”)) %>% select(-count)
port_y_train <- port_y %>% filter(date < ‘2022-12-31’)
trade_func <- function(data, prop, rate, sent) { port_y_norm <- data %>% #normalizing indicators mutate(mean_rating = mean_rating/5, sentiment = sentiment/5) %>% transmute(date, company, close = Close, y, prop_postive, mean_rating, sentiment, lag_close) %>% drop_na() %>% mutate(comb_sent = propprop_postive + ratemean_rating + sentsentiment, price_norm = 1 / lag_close) %>% group_by(y) %>% #weighted sentiment mutate(weight_sent = comb_sent / sum(comb_sent)) %>% #adj for close prices in the prior year mutate(weight = weight_sentprice_norm/sum(price_norm*weight_sent)) %>% arrange(company, date) %>% group_by(company) %>% mutate(previous_weight = lag(weight), # previous weight trades = case_when( date == ‘2015-12-31’ ~ 1, # Starting strategy is.na(previous_weight) ~ NA_real_, # Handle the first row weight < previous_weight ~ -1, # Selling weight > previous_weight ~ 1, # Buying TRUE ~ 0 # No change )) %>% ungroup() %>% mutate(position = 4)
weighted_returns <- port_y_norm %>% mutate(w_close = weight*close) %>% group_by(date) %>% summarise(port_value = sum(w_close)) %>% ungroup() %>% mutate(logRet = log(port_value/lag(port_value))) %>% drop_na() %>% summarise(logRet_sum = sum(logRet), var = var(logRet)) %>% mutate(prop = prop, rate = rate, sent = sent)
return(weighted_returns)
}
trade_func(port_y, 0, 1, 0)
out <- expand.grid( prop = seq(from = 0, to = 1, by = .02), rate = seq(from = 0, to = 1, by = .02), sent = seq(from = 0, to = 1, by = .02)) %>% mutate(total = prop + rate + sent) %>% filter(total == 1)
library(foreach) library(doParallel)
n_cores <- detectCores() - 1
cl <- makeCluster(n_cores) registerDoParallel(cl)
res <- foreach( combo = iter(out, by = ‘row’), .combine = rbind, .packages = c(“dplyr”, “tidyverse”, “tidyquant”, “PerformanceAnalytics”) ) %dopar% {
prop <- combo\(prop rate <- combo\)rate sent <- combo$sent
trade_func(data = port_y_train, prop = prop, rate = rate, sent = sent) }
stopCluster(cl)
rebal_func <- function(data) { port_y_norm <- data %>% transmute(date, company, close = Close, y, lag_close) %>% drop_na() %>% mutate(price_norm = 1 / lag_close) %>% group_by(y) %>% #adj for close prices in the prior year mutate(weight = price_norm/sum(price_norm)) %>% arrange(company, date) %>% group_by(company) %>% mutate(previous_weight = lag(weight), # previous weight trades = case_when( date == ‘2015-12-31’ ~ 1, # Starting strategy is.na(previous_weight) ~ NA_real_, # Handle the first row weight < previous_weight ~ -1, # Selling weight > previous_weight ~ 1, # Buying TRUE ~ 0 # No change )) %>% ungroup() %>% mutate(position = 4)
weighted_returns <- port_y_norm %>% mutate(w_close = weight*close) %>% group_by(date) %>% summarise(port_value = sum(w_close)) %>% ungroup() %>% mutate(logRet = log(port_value/lag(port_value))) %>% drop_na() %>% summarise(logRet_sum = sum(logRet), var = var(logRet))
return(weighted_returns)
}
rebal_func(port_y)
port_y_norm <- port_y %>% #normalizing indicators mutate(mean_rating = mean_rating/5, sentiment = sentiment/5) %>% transmute(date, company, close = Close, y, prop_postive, mean_rating, sentiment, lag_close) %>% drop_na() %>% mutate(comb_sent = 0prop_postive + 1mean_rating + 0sentiment, price_norm = 1 / lag_close) %>% group_by(y) %>% #weighted sentiment mutate(weight_sent = comb_sent / sum(comb_sent)) %>% #adj for close prices in the prior year mutate(weight = weight_sentprice_norm/sum(price_norm*weight_sent)) %>% arrange(company, date) %>% group_by(company) %>% mutate(previous_weight = lag(weight), # previous weight trades = case_when( date == ‘2014-12-31’ ~ 1, # Starting strategy is.na(previous_weight) ~ NA_real_, # Handle the first row weight < previous_weight ~ -1, # Selling weight > previous_weight ~ 1, # Buying TRUE ~ 0 # No change )) %>% ungroup() %>% mutate(position = 4)
weighted_returns <- port_y_norm %>% mutate(w_close = weight*close) %>% group_by(date) %>% summarise(port_value = sum(w_close)) %>% ungroup() %>% mutate(logRet = log(port_value/lag(port_value))) %>% drop_na() %>% summarise(logRet_sum = sum(logRet), var = var(logRet)) %>% mutate(prop = 0, rate = 0, sent = 1)
weighted_close <- port_y_norm %>% mutate(w_close = weight*close)
weighted_close %>% mutate(Year = year(date)) %>% ggplot(aes(y = port_value, x = y, col = company)) + geom_line()
```